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APPARATUS AND METHOD FOR COMPENSATING FOR 
RESPIRATORY AND PATIENT MOTION DURING TREATMENT 



Related Application 



This application is a continuation in part of US Patent Application Serial No. 
5 09/270,404, filed March 16, 1999 entitled "Apparatus and Method for Compensating for 
Respiratory and Patient Motion During Treatment" which is owned by the same assignee 
as the present invention and is incorporated herein by reference. 



Background of the Invention 

This invention relates generally to an apparatus for improving the accuracy and 
10 efficacy of surgical treatments and more particularly to locating a target region to be 
treated and tracking the motion of the target region due to respiratory and other patient 
motions during the treatment. 

Various different treatments may accurately track the motion of a target region in 
order to apply the treatment to the target region. In radiation therapy and radiosurgery, 

15 for example, a tumor may be destroyed by a beam of ionizing radiation which kills the 
cells in the tumor. The problem is that the tumor may move during treatment, especially 
due to the breathing motion of the patient. Such respiratory motion is difficult to track 
using external sensors, since the extent and direction of the internal breathing motion of 
the patient cannot be seen with traditional imaging devices. The breathing and other 

20 motion of the patient means that it is more difficult to focus the radiation on the tumor 
which means that the treatment may be less effective and healthy tissue may be 
unnecessarily damaged. 
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The goal of radiosurgery is to give a very high dose of radiation to the tumor 
only, while protecting surrounding healthy tissue as much as possible. Although 
radiosurgery has been applied with dramatic success to brain tumors, the extension of 
this technique to tumors outside the head or neck areas has eluded easy solutions. The 

5 main reason for this difficulty has been the problem of accurate target localization (i.e., 
accurate tracking of the motion of the target). In particular, breathing motion and other 
organ and patient motion make it difficult to track the target tumor with high precision. 
Thus, in the presence of breathing motion, for example, it is difficult to achieve the goal 
of providing a high dose of radiation to the tumor while protecting surrounding healthy 

10 tissue. Therefore radiosurgery is currently applied nearly exclusively to brain tumors. 
Conventional systems can only move the radiation beam along circular arcs in space so 
that irregular breathing motions cannot be easily followed since these breathing motions 
may not occur along the axis of the circular arcs traced by the radiation beam. 

Another radiosurgery technique uses a mechanical robotic device having six 
15 degrees of freedom that targets a radiation beam as disclosed in U.S. Patent No. 
5,207,223 to Adler. The robotic device permits the radiation treatment beam to be 
accurately positioned to apply the treatment beam directed to the target region. A 
method for neurosurgical navigation is disclosed in U.S. Patent No. 5,769,861. This 
method relates to finding fixed targets, such as a brain tumor, but does not address 
20 tracking the motion of a target organ, such as lung due to breathing, with respect to the 
skin surface, or tracking the motion of internal abdominal organs with respect to 
externally visible motion. A fiducial that may be implanted into the human body so that 
it is detectable by an imaging system is also disclosed in which the fiducial implant is 



WO 02/19908 PCT/US01/28164 

3 

implanted into the bone or organs of the human body. This fiducial implant permits 
internal structures of the human body to be analyzed, but does not attempt to compensate 
for motion of a target organ which moves throughout the respiratory cycle. Thus, it is 

* 

desirable to provide an apparatus and method for compensating for respiratory and other 
5 patent motion in radiation treatment and it is to this end that the present invention is 
directed. 

Summary of the Invention 

In accordance with the invention, an apparatus and method for compensating for 
breathing and other motion of a patient is provided which combines internal markers 

10 placed on the target organ with one or more external sensors to accurately track the 
position and motion of a moving target region, such as an internal organ. In particular, 
the position of the internal markers, determined periodically by x-rays, may be combined 
with the position of the external markers/sensors. The internal markers may be imaged 
only periodically since an invasive technology, such as x-rays, are needed to image the 

15 internal markers. The external continuous or real-time sensor, which may be an external 
marker, determines external motion during treatment. Thus, the position of the target 
organ may be precisely determined by the position of the internal markers when the 
internal markers are periodically imaged and may be determined based on the external 
sensor data while the internal markers are not being imaged. The position and motion of 

20 the internal markers relative to the external sensors are determined so that the position of 
the internal markers and therefore the target organ may be accurately determined by the 
position of the external sensor. Thus, the position of the target organ may be accurately 
determined throughout the medical procedure being performed. 
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The internal markers may be imaged using a number of different imaging 
technologies, including x-rays, nuclear magnetic resonance, ultrasound and other 
technologies which permit markers inside of the body of the patient to be imaged. 
Alternatively, three dimensional ultrasound images may be used to establish the location 
5 of the internal target region in lieu of discrete fiducials. The position of the external 
sensor may also be determined using a number of different technologies including 
infrared imaging, visual imaging, magnetic localization, the measurement of respiration, 
and any other type of technology which permits the external markers to be imaged. In 
addition to using external sensors (i.e., external fiducials may not be used), it is also 
10 possible to visually image a body surface which is then correlated to the internal 
fiducials. 

Thus, in accordance with the invention, an apparatus for performing treatment on 

■ 

an internal target region while measuring and in some cases compensating for breathing 
and other motion of the patient is provided. The apparatus comprises a first imaging 

15 device for periodically generating positional data about the internal target region and a 
second imaging device for continuously generating positional data about one or more 
external markers attached to the patient's body. The apparatus further comprises a 
processor that receives the positional data about the internal target region and the 
external sensor readings/measurements in order to generate a correspondence between 

20 the position of the internal target region and the external marker or sensor readings and a 
treatment device that directs the treatment towards the position of the target region of the 
patient based on the positional data of the external markers or sensor readings. An 



• 
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apparatus for compensating for motion of a patent during treatment is also disclosed as 
well as a method for compensating for motion of the patient. 



In accordance with another embodiment of the invention, no fiducials are 
attached to the target, the target is delineated in x-ray images (manally or semi- 
5 automatically), and the target's position is subsequently matched to the position of the 
tumor in preoperative tomographic images. In addition, motions of the target region 

■ 

resulting from pulsation effects can be compensated for via a sensor by recording both 
pulsation data and time points of pulsation data acquisition or both pulsation and 
respiratory motions may be compensated for simultaneously. In addition, the 
10 deformation or squeezing of the target region may be determined and that deformation 
may be compensated. 



In accordance with yet another aspect of the invention, a single image of the 
patient may be used to generate a model and then the patient is instructed to return to the 
position within the respiration cycle corresponding to this single point. In addition, the 
15 motion model in accordance with the invention may be continuously updated. In 
addition,a general model for a particular motion may be determined. In addition, the 
time lag between a command and the beam actually being at that position may be 
compensated for. Finally, respiratory motion may be distinguished from other motion. 



Brief Description of the Drawings 
20 Figure 1 is a diagram illustrating a conventional radiation treatment device; 

Figure 2 is a diagram illustrating more details of the radiation treatment device; 
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Figure 3 is a diagram illustrating more details of the radiation treatment device; 

Figure 4 is a block diagram illustrating the radiation treatment device; 

Figure 5 is a diagram illustrating internal markers on a target organ moving as the 
target organ moves; 

5 Figure 6 is a diagram illustrating one or more internal markers attached to the 

target organ being imaged by x-ray devices; 

Figures 7 A - 7D are diagrams illustrating the imaging of the internal markers in 
accordance with the invention; 

Figure 8 is a diagram illustrating the internal markers in combination with an 
10 external marker to track the motion of the target region in accordance with the invention; 

* 

Figures 9A - 9D are diagrams illustrating the reduction in the safety margin in 
accordance with the invention; 

Figure 10 is a flowchart illustrating a method for compensating for breathing and 
other motion in a radiosurgical device; 

15 Figure 1 1 is a flowchart illustrating a method for correlating the internal and 

external markers in accordance with the invention; 

Figure 12 illustrates an example of an ultrasound system in accordance with the 
invention; 
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Figure 13 is a flowchart illustration an example of an ultrasound treatment 
method in accordance with the invention; 

Figure 14 is a diagram illustrating the process for manually marking the target 
using ultrasonic image pairs in accordance with the invention; and 

5 Figure 1 5 is a diagram illustrating the automatic alignment of the marked target 

region with a preoperative image of the target region. 

Detailed Description of a Preferred Embodiment 

The invention is particularly applicable to an apparatus and method for 
compensating for breathing and other patient motion during radiation treatment 

10 (radiosurgery) and it is in this context that the invention will be described. It will be 
appreciated, however, that the apparatus and method in accordance with the invention 
has greater utility, such as to other types of medical procedures with other types of 
medical instruments, such as positioning biopsy needles, ablative, ultrasound or other 
focused energy treatments, or positioning a laser beam for laser beam treatment. Prior to 

15 describing the invention, a typical radiosurgery device will be described to provide a 
better understanding of the invention. 

Figures 1 - 4 are diagrams illustrating an example of a stereotaxic radiation 
treatment device 10. The radiation treatment device 10 may include a data processor 12, 
such as a microprocessor, and a disc or tape storage unit 13 (shown in Figure 4) which 
20 may store a three dimensional image of a patient 14. The three dimensional image may 
be loaded into the data processor, if not already there, to compare the three dimensional 
image to images generated during the surgical procedure. The three dimensional image 
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may be generated by various conventional techniques such as computer aided 
tomography (CAT) scan or magnetic resonance imaging (MR). The radiation treatment 
device 10 may also include a beaming apparatus 20 which, when activated, emits a 
collimated ionizing beam directed at a target region 1 8 (shown in Figure 2). The 
5 collimated surgical ionizing beam may have sufficient strength to cause the target region 
to become necrotic. A variety of different beaming apparatus may be used which 

■ 

generate an ionizing radiation or heavy particle beam such as a linear accelerator, a 
synchrocyclotron or preferably an x-ray linear accelerator. Such an x-ray beaming 
apparatus is commercially available. The beaming apparatus may be activated by the 
10 operator throwing a switch 23 at a control console 24 connected to the beaming 
apparatus 20 by a cable 22. 

The radiation treatment device 10 may also include an apparatus for passing a 
first diagnostic beam 26 and a second diagnostic beam 28 through the region previously 
imaged by the three-dimensional image. The diagnostic beams are positioned at a 

15 predetermined non-zero angle with respect to each other, such as being orthogonal as 
shown in Figure 2. The diagnostic beams may be generated by a first x-ray generator 30 
and a second x-ray generator 32, respectively. A single image receiver 34 or a first and a 
second image receiver 34, 36 (as shown) may receive the diagnostic beams 26, 28 to 
generate an image from the diagnostic beams which is fed into the microprocessor 12 (as 

20 shown in Figure 4) so that the diagnostic images may be compared to the three- 
dimensional image. 

The radiation treatment device 10 may also include a device for adjusting the 
relative positions of the beaming apparatus 20 and the patient 14 so that the ionizing 
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beam is continuously focused on the target region 18. In the radiation treatment device 
shown in Figure 1 , the positions of the beaming apparatus and the patient may be altered 
with six degrees of freedom by a gantry 40 and a moveable operating table 38 with a 
tilting top 44. The positions of the beaming apparatus relative to the patient may also be 
5 accomplished by using a processor controllable robotic arm mechanism 46 as shown in 
Figure 3 which has six axes of motion. The robotic arm mechanism permits the beaming 
apparatus to be moved freely about the patient's body including up, down, longitudinally 
along or laterally along the body of the patient. 

Figure 4 is a block diagram of the radiation treatment device 10 including the 
10 microprocessor 12, the tape drive 13, the beaming apparatus 20, the robotic arm 46 or the 
gantry 40, the x-ray cameras 30, 32, 34 and 36, and the operator control console 24 as 
described above. In addition, the device 10 may include safety interlocks 50 to ensure 
that the beaming apparatus is not activated accidentally. The device 10 may also include 
an operator display 48 for tracking the progress of the treatment and controlling the 
15 treatment. Any further details of the radiosurgery device may be found in U.S. Patent 
No. 5,207,223 which is owned by the assignee of this application and which is 
incorporated herein by reference. 

To accurately target the area to be irradiated in radiation therapy or radiosurgery, 
it is necessary to determine with high precision where the target is located during 
20 treatment. The above radiosurgery device may be ideally used for the treatment of brain 
or head tumors since the brain is fixed with respect to a rigid skull. The radiosurgery 
device may also be used with other fixed target regions in which it is easy to ensure that 
the ionizing beam strikes the target region, but not surrounding healthy tissue. If the 
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target is adjacent to the diaphragm, however, the target will move during treatment due 
to breathing of the patient. The lung and other organs will move when the patient 
breathes or whenever the patient moves during the treatment. Therefore, it is desirable 
to provide an apparatus and method that follows a moving target region during a variety 
5 of different treatments, including radiation treatment. The apparatus may compensate for 
movements of the target region caused by breathing of the patient as well as movement 
of the target region caused by other movements of the patient. 

■ 

In accordance with the invention, to determine the position of an internal moving 
target region such as an internal organ, external and internal markers (landmarks) may be 

10 used, as described below, and a model of their relative motions based on a series of 

images is determined prior to treatment. During treatment, little information is available 
on the placement of the internal landmarks except when the internal markers are 
periodically imaged using invasive devices, such as x-rays. However, the position of the 
external markers or a video image of the chest and/or abdomen may be determined with 

15 high precision and speed. Similarly, external sensors can provide measurement data in 
real time, i.e., at very high speed. Thus, the position of the external landmarks may be 
used in real time during treatment by inferring the placement of the internal (exact) 
markers by referencing the pre-operative model of the relative motion of internal and 
external markers. For verification, the placement of the internal markers can be 

20 determined periodically during treatment. An apparatus that compensates for breathing 
and other patient motion in accordance with the invention will now be described. 

Figure 5 is a diagram illustrating a set of internal markers 152 in accordance with 
the invention placed on a target organ 1 5 1 within a body 1 50 of the patient. The moving 
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target organ 1 5 1 may be, for example, an organ near the diaphragm such as a lung or a 
liver which may move as the patient moves or as the patient inhales or exhales. In 
accordance with the invention, it is desirable to be able to track the motion of the target 
organ so that the treatment, such as ionizing radiation, is applied to the target organ and 
5 not to the healthy surrounding tissue. To track the movement of the target organ 1 5 1 , the 
one or more internal markers 152 may be attached to or placed near various locations on 
or near the target organ 151. Then, as the target organ moves, the internal markers also 
move as shown by arrows 154. From the placement of the internal markers, it is possible 
to precisely determine the position of the target organ. In a preferred embodiment, more 
10 than one internal marker may be used in order to measure the movement of different 
areas of the target organ and the internal markers may be made of gold so that, although 
the internal markers are not visible outside of the body, the internal markers may be 
viewed using an imaging technique, which may preferably be stereotaxic x-ray imaging, 
but may also be ultrasound. 

15 To track the moving target, the therapeutic beam may be moved by a robot arm 

or the patient couch may be moved. In the alternative, the beam may be moved by 
moving the leaves of a multileaf collimator or the beam may be switched off and on 
periodically such that the target is in the beam path, whenever the beam is on. 

Figure 6 is a diagram illustrating one or more internal markers 152 attached to the 
20 target organ 151 being imaged by a stereotaxic x-ray device. As shown in Figure 6, the 
internal markers 152 on the target organ 151 may be imaged by a first x-ray source 160 
and a second x-ray source 162 which are positioned at some predetermined angle with 
respect to each other similar to the diagnostic x-ray beams shown in Figures 1-3. The x- 
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ray sources may generate a first and second diagnostic x-ray beam 164, 166 which pass 
through the target organ 151 near the internal markers 152 and are received by a first and 
second x-ray receiver 168, 170, respectively, which receive the x-ray beams and generate 
an electrical signal corresponding to the received x-rays. The stereotaxic x-ray device 
5 permits the precise location of the internal markers 152 to be determined by analyzing 
the images generated. 

Figures 7A - 7D are diagrams illustrating examples of the x-ray images of a 
target organ that include the internal markers 152 in accordance with the invention. 
Figures 7 A and 7C show the same x-ray image with the internal markers 1 52 
10 unenhanced and with the internal markers 1 52 being computer enhanced, respectively. 
Similarly, Figures 7B and 7D also illustrate the same x-ray image with unenhanced 

m 

internal markers and computer enhanced internal markers, respectively. Thus, the 
stereotaxic x-ray imaging permits the precise location of the internal markers to be 
determined. The problem is that, using the stereotaxic x-ray device, internal marker 

15 positions may be determined only at predetermined intervals during treatment. In 

particular, the interval between imaging of the internal markers is necessary in order to 
limit the patient's exposure to the radiation and because the treatment beam can not be 
activated while the x-ray diagnostic imaging occurs. However, determining the exact 
position of the target organ periodically is not sufficient in order to accurately 

20 compensate for breathing and other motions of the patient. Therefore, one or more 
external markers may be placed on the skin of the patient near the target organ in 
accordance with the invention as will be described with reference to Figure 8. 
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Figure 8 is a diagram illustrating the target organ 151 within a patient's body 150 
having internal markers 152 in combination with one or more external markers 180 
attached to the skin of the patient in accordance with the invention. The one or more 
external markers 180 that are attached to the skin of the patient permit the motion 182 of 
5 the abdomen or chest wall to be determined. In the example of the breathing of a patient, 
the external marker may track the external motion as the patient inhales and exhales. 
The external markers 180 may be automatically tracked with a number of optical 
methods, such as infrared or visible light, and the position of the external marker may be 
determined more than 60 times per second. The external markers may also be attached 
10 to a belt, the skin itself, a flexible ring or a vest which fits around the waist of the patient. 

If only external markers are used to compensate for the motion of the patient, 
however, they cannot accurately reflect the internal motion of the target organ since the 
target organ may move a small amount while the external marker may move a larger 
amount and vice versa. The external markers are not sufficiently precise to compensate 

15 for the motion of the patient. Therefore, the combination of the internal markers and the 
external markers is necessary in order to accurately track the motion of the target organ. 
Thus, the periodic x-ray imaging of the internal markers is synchronized with the 
continuous optical tracking of the external markers to provide accurate tracking of the 
motion of the target organ. In order to synchronize the motion of the internal and 

20 external markers, it is necessary to determine the relationship between the positions of 
the internal and external markers which may occur at the start of the treatment process 
and will be described below with reference to Figure 10. 



WO 02/19908 PCTAJS01/28164 

14 

When some movement of the target organ is detected, the treatment system, such 
as the radiosurgery device described above, may compensate for the movement in a 
number of different ways. For example, the treatment system may move the treatment 
device, such as the beaming apparatus 20, relative to the patient or vice versa. The 

5 treatment system may also move a shaping or collimating device that is within the path 
of the treatment device to change the characteristics of the treatment device. The 
treatment system may also only activate the treatment device when the target organ is 
within the treatment path or block the treatment device when the target organ is not 
within the treatment path. Now, one of the benefits of the motion compensation 

10 apparatus in accordance with the invention will be illustrated and described. 



Figures 9A - 9D are diagrams illustrating one benefit of the motion compensation 
apparatus in accordance with the invention. In particular, Figure 9A shows a volume of 
target region, such as a tumor, to be treated 200 without a safety margin. In particular, 
the volume may have no safety margin provided that the position of the target region to 

15 be treated is precisely known so that healthy tissue is not damaged. If the position of the 
target region cannot be exactly determined, such as for a moving target organ due to 
breathing and other patient motion, the volume 200 requires a safety margin 202 as 
shown in Figure 9B. The problem with the safety margin is that the required volume of 
irradiation increases very rapidly with the diameter of the target. For example, for a 

20 spherical target the ratio between the diameter of the target and the required dose is 
cubic. The safety margin 202 for a typical radiosurgery device is shown in Figure 9C. 
Figure 9D shows the much reduced safety margin 202 which is possible due to the 
motion compensation apparatus and method in accordance with the invention. A 
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reduction of the safety margin by a factor of two results in a volume reduction of the 
dose by a factor of eight. Thus, the unwanted dose to healthy tissue may be reduced by a 
factor of four while the dose to the target organ or tumor may be doubled. For a large 
variety of cancer cases with particularly grim prognosis, the motion compensation 
5 apparatus in combination with typical radiosurgery devices can greatly improve the 
efficacy of the treatment. Now, a method for motion compensation during treatment in 
accordance with the invention will be described. 

Figure 10 is a flowchart illustrating a method 210 for compensating for breathing 
and other motion of a patient during treatment, such as with a radiosurgical device. The 

10 first few steps in the method may be performed at a time prior to the actual treatment of 
the patient. In particular, a surgeon may attach a set of internal markers in the proximity 
of or within the target organ during a short surgical procedure in step 212 and then, just 
prior to treatment, the surgeon may attach a set of external markers to the chest or 
abdominal wall of the patient near the target organ in step 214. Next, the processor of 

15 the radiosurgery device correlates the position of the internal markers and the external 
markers in step 216 just prior to starting the treatment of the patient. The method for 
correlating the internal markers with the external markers is described below with 
reference to Figure 1 1 . Once the positions of the internal and external markers have 
been correlated, the treatment of the patient may begin. 

20 First, the apparatus determines if the total elapsed time since the last time the 

internal markers were imaged is equal to a predetermined number of seconds. The 
predetermined number of seconds is preferably between two and ten seconds and more 
preferably about ten seconds. If the total elapsed time is equal to the predetermined 
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number of seconds, then the treatment beam is deactivated and the internal markers are 
imaged using, for example, stereotaxic x-ray imaging in step 220. Next, the total 
elapsed time is reset to zero and the method returns to step 218. Thus, in accordance 
with the invention, the internal markers are imaged every predetermined number of 
5 seconds. Returning to step 2 1 8, if the total elapsed time is not equal to the 

« 

predetermined number of seconds, then the external markers are optically tracked in step 
224 while the treatment beam is activated in step 226. The external markers may be 
tracked so that position data is provided to the processor of the radiosurgery device as 
much as sixty times per second. The processor may then correlate the position of the 
10 external markers with the internal markers and generate positional data about any change 
in the position of the target organ. Thus, between the periodic imaging of the internal 
markers, the position of the external markers is used to track the position of the target. 



When some movement of the target organ is detected, the treatment system, such 
as the radiosurgery device described above, may compensate for the movement in a 
15 number of different ways. For example, the treatment system may move the treatment 
device, such as the beaming apparatus 20, relative to the patient or vice versa. The 

» 

treatment system may also move a shaping or collimating device into the path of the 
treatment device to change the characteristics of the treatment beam. The treatment 
system may also . only activate the treatment device when the target organ is within the 
20 treatment path or block the treatment device when the target organ is not within the 
treatment path. Now, a method for correlating the positions of the internal and external 
markers in accordance with the invention will be described. 
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Figure 1 1 is a flowchart illustrating a method 230 for correlating the positions of 
the internal and external markers in accordance with the invention. Several steps of the 
method occur during the pre-operative process while several steps occur during the 
actual treatment. In particular, in step 232, a series of time sequence images are 
5 generated for both the internal and external markers throughout the respiratory cycle so 
that a plurality of points corresponding to the internal and external markers are generated 
during the pre-operative phase. The plurality of points corresponding to the external 
markers and the internal markers may each be referred to as a point cloud. Next, the 
processor in the radiosurgery device may fit a curve to the points generated for the 
10 internal markers and a separate curve to the points generated by the external markers in 
step 234. These curves permit the positions of the external and internal markers to be 
correlated to each other. 

During the actual treatment, the system generates a position, x, of the external 
markers by a technique, such as infrared imaging, at a particular time and that position, 

15 x, is fit to the previously generated curve of the external marker positions in step 236. 
Next, in step 238, a position, y, of a point on the internal marker curve which 
corresponds to the position, x, is determined by comparing the two curves which is a 
process known as interpolation. This process may be performed for each marker. Using 
this method, the position of the external markers day be correlated to the position of the 

20 internal markers which permits the system to accurately determine the amount of 

movement of the target organ without actually imaging the internal organ. Another way 
to perform the correlation of the positions of the internal and external markers is to use a 
neural network trained to perform interpolation or other known mathematical 
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interpolation methods for establishing the correspondence between the internal and 



external markers after having computed the point clouds. 



Now, various other embodiments of the method for compensating for breathing 
and other patient motion in accordance with the invention will be described. In all of 
5 these embodiments, the respiratory and diaphragmatic excursion may be limited and 
minimized by binding the abdomen or compressing the abdomen. In a first embodiment, 
one or more small metal markers (also known as landmarks) are attached to the target 
organ before treatment. There may be three or four metal markers with possibly distinct 
shapes or sizes, which may be, for example, small gold beads. The exact position of 

10 these internal markers is determined by two x-ray cameras, which acquire a stereo image 
of the target site. There may also be one or more infra-red probes which are attached to 
the patient's skin surface. The infra-red probes give a very accurate and high speed 
position reading, but they only show the surface of the patient's body. In this 
embodiment, internal imaging of the internal markers and external imaging of the 

15 external markers (i.e., x-ray imaging and infrared imaging) are combined. In particular, 
prior to treatment, a series of images with both modalities (i.e., x-ray and infrared, 
respectively) is obtained. For these images, the time of image acquisitions is recorded, 
or at least the images with both modalities are acquired simultaneously so that the time 
of image acquisition does not vary by more than approximately 0.01 sec. In this way, a 

20 series of pre-operative images of both external and internal landmarks is acquired where 
each image has a time-stamp. These series of images determines a model of the relative 
motion between internal and external landmarks as described above. 
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During the actual operation, it is difficult to obtain x-ray images more than once 
every predetermined number of seconds due to concerns about exposing the patient to 
too much radiation and due to the fact that the treatment beam cannot operate when x-ray 
imaging is being done. The x-ray imaging alone would therefore be too slow to follow 

5 breathing motion with high precision. Therefore, the external landmarks on the skin 
surface, as seen by the infrared system, are used for intra-operative localization, where 
we continuously reference the previously computed model of relative motions of the 
internal and external markers. This allows the exact placement of the internal landmarks 
(gold beads) to be predicted at time points where no x-ray images are available. Now, a 

10 second embodiment of the method will be described. 

In a second embodiment, there are situations in which it is desirable to 
compensate for respiratory motion as well as to compensate for pulsation effects. These 
pulsation effects may cause motions of the patient in much the same way as respiration 
does. In accordance with the invention, a strain gauge or some other type of pulsation 

15 measurement device captures the current state of pulsation of the patient. A device for 
measuring the pulsation movement may, for example, be a device for recording 
electrocardiograms, or similar devices used for monitoring patient-specific data in an 
intensive care unit. Another device may be fit to a patient's wrist and detect the state of 
aortic pulsation via tactile information. As above, one or more time stamps may be used 

20 to obtain a pulsation motion pattern representing the internal target motion caused by the 
pulsation effects. Thus, by using a sensor for observing the external effects of 
respiratory motion, we can compensate for both respiratory motion and pulsation motion 
at the same time. Now, a third embodiment of the invention will be described. 
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In a third embodiment of the method in accordance with the invention, no 
internal landmarks attached to the target organ are used. Instead, an ultra-sound camera 
is used to acquire the pre-operative image series, again in combination with an infra-red 
tracking system. The infra-red system in this embodiment establishes both the position 
5 of the external landmarks and the position of the (movable) ultra-sound camera, which 
must be moved by a human operator during this pre-operative phase. During the pre- 
operative phase, the ultra-sound images may be analyzed manually or semi-automatically 
in order to locate the target. During treatment, the external landmarks (infra-red probes) 
are used to compensate for the motion of the target organ since the motion model we 
10 have established allows the determination of the position of the internal target organ 
from the position of the external markers. 

Furthermore, there are situations, where the attachment of markers to an anatomic 
target is difficult, undesirable or costly. In many cases, such fiducials are necessary, 
because the target tumor itself is not well visible in x-ray images. However, in some 
15 cases the tumor is partly visible in an x-ray image at least for a human operator. In such 
cases, the method described above may be used without placing such internal markers. 
This can, for example, occur for larger peripheral tumors of the lung, or also for certain 
tumors of the liver. 

If the tumor is visible in x-ray images, the method may proceed as follows. 
20 Instead of using internal markers, the target is manually or semi-automatically delineated 
in the x-ray images while it is displayed on a computer screen, for example. From this 
delineation, one can compute the exact placement of the target tumor at the time point 
when the x-ray image were taken. To perform this computation, the two-dimensional 
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contour of the tumor, as determined by the delineation, is matched to the three- 
dimensional shape of the target, as known from the pre-operative tomography. The time 
stamps of the external markers are now used as above to form a model for the correlation 
between the internal and external motion. To help the process of delineation on the 
5 computer screen, the contour of the three-dimensional target shape may be projected 
onto the x-ray image displayed on the screen so that only alignment is necessary. 

In the case of ultrasound being used a one of the sensors, the problem of placing 
the ultrasound camera on the patient's body arises since the ultrasound signal should not 
travel through air. In addition, the ultrasound cameras must be moved during treatment 

10 by an experienced operator to ensure (1) that the tumor is within the field of view of the 
camera; (2) that the tumor is visible with sufficient clarity in the image; and (3) the 
signal should not travel in air. To accomplish this, a movable ultrasound camera, placed 
on a cushion filled with liquid either performs periodic or regular motion, or the motion 
of the ultrasound camera is controlled by an operator from outside the operating room. 

15 The operator moving the ultrasound camera must remain outside the treatment room, due 
to the presence of radiation in the treatment room. The step of moving the ultrasound 

■ 

camera from outside the room can be accomplished by attaching ultrasound sensors to a 
small robot arm. To move the ultrasound camera, the operator moves this robot arm via a 
control wand or joystick, or a computer mouse pointer. Now, more details of the 
20 ultrasound treatment system will be described. 

Figure 12 is a diagram illustrating an example of a ultrasonic treatment system 
250 in accordance with the invention wherein an ultrasound sensor may be used for 
detennining the position of a target region in a patient. In this example, the ultrasonic 
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sensor may be remotely controlled by an operator. In particular, as shown, a patient 
table 252, a treatment robot 254 that generates the therapeutic beam and an ultrasound 
guidance system 356 may be located in an operating room 358. Then, an ultrasound 
camera and motion control system 260 may be located remote from the operating room 
5 due to the radiation present in the operating room. In operation, an ultrasound specialist 
may move the ultrasound camera from outside of the operating room. Now, a method 
for marking a target region using the examplary ultraound system will be described. 

Figure 13 is a flowchart illustrating a method 270 for marking and treating a 
target region using ultrasound in accordance with the invention. In particular, in step 

10 272, a model for the correspondance between the internal motion and the external motion 
is determined as described above in more detail. In step 274, a sensor reading to 
determine the external motion is taken and in step 276, an ultrasound image for 
determining the internal motion is taken. In step 278, it is determined by the ultrasound 
operator if the target is visible in the ultrasound image. If the target region is not visible 

15 in the ultrasound image, then the position of the internal fiducial markers is determined 
in step 280. If the target region is visible in the ultrasound image, then the target region 
is marked in the ultrasound images in step 282. In step 284, based on the fiducial 
position or based on the target region marked in the ultrasound images, the position of 
the target region is determined. In step 286, the radiation beam or the patient couch is 

20 moved to aim the beam at the new target position. Thus, the ultrasound images may be 
used to locate the current position of the target region or the position of the fiducials may 
be used. Figure 14 illustrates a target region 290 being imaged by two ultrasound images 
wherein each ultrasound image shows a cross-section of the visible target region. Figure 



* 
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15 illustrates those ultrasound cross-sections being used to determine the position of the 
target region by matching the cross-sections with the preoperative target region image. 
To prevent the small robot (carrying the ultrasound camera) from applying undesired 
forces (too much pressure) on the patient's body, a force sensor may be attached to the 
5 small robot. This force sensor stops or reduces a motion of the robot directed into the 
patient's body. 

Another embodiment is very similar to the third embodiment except that a device 
for measuring the air flow may be used instead of the ultrasound camera. The device 
may be a mouthpiece that records the direction and volume of airflow and correlates 

10 these measurements with the location of the internal fiducials or any other mechanism 
for detecting the location of the internal organs. In this embodiment, a reference position 
of the target organ, such as a lung, such as at full exhalation or at full inhalation or any 
intermediate respiratory state may be used to correlate the current respiratory state to a 
state imaged prior to the treatment so that the motion and position of the target organ 

15 may be determined during the treatment and the position of the treatment device may be 
moved based on the determined motion of the target organ. Now, a fifth embodiment of 
the invention will be described. 

In yet another embodiment of the invention, a slightly different technique is used. 
In particular, during most radiation treatments, the patient is awake and conscious so that 
20 it is often difficult to determine whether a motion observed by real-time tracking of 
external markers is indeed due to breathing and not to other small movements of the 

* 

patient's body. Such other movements of the body may be caused, for example, by 
sneezing or other sudden motions. To detect and accurately track these other motions, a 
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pair of x-ray cameras in addition to the ultra-sound camera described above may be used. 
In this embodiment, the ultra-sound camera is only used before the operation to 
determine the correlation between the target motion and the motion of external 
landmarks as described above. Thus, a series of pre-operatiye images is again acquired 
5 to determine the relationship between the motion of the patient's skin surface and the 
target organ. During treatment, the x-ray cameras may be used to determine sudden 
motion of the patient based on well known computer methods for automatically finding 
bony landmarks in x-ray images. These x-ray images may be compared to pre-operative 
tomographic images (CT or MR images) to determine sudden movement of the patient. 

10 In more detail, it may be necessary to distinguish between patient shift, caused by 

voluntary motion or by sneezing on the one hand, and respiratory motion on the other 
hand. These two types of motion must be processed in different ways. For example, a 
voluntary sideward motion of the patient should result in a shift of the entire deformation 
model whereas normal respiration should not shift the entire deformation model and only 

15 the beam should be moved accordingly. To distinguish the two types of motion, the 
system may, at each time point at which both an internal sensor and an external sensor 
reading is available, calculate the inferred internal position from the current deformation 
model. The system may then compare the actual position to the inferred position and 
any deviation exceeding a fixed threshold value will lead to a recomputation of the entire 

20 deformation model (implying a detection of a patient movement instead of respiration). 
After the recomputation, the treatment may resume. Thus, in accordance with the 
invention, the system may detect a patient movement other than respiration and therefore 
stop to recompute the deformation model before recommencing the treatment. If the 
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movement is respiration movement, the treatment continues since that motion is 
compensated for by the deformation model. 

In yet another embodiment of the invention, the system described may be used to 
track deformation and/or squeezing of a target region. For example, in some cases the 
5 target region, such as a tumor, can be squeezed/deformed by respiration to some extent. 
In this case, the system may detect such squeezing/deformation by the motion of the 
internal fiducials as described above. As described above, interpolation between the 
different relative placements of the internal fiducials allows the system in accordance 
with the invention to determine the current deformation state of the tumor so that the 
10 treatment can take into account the current deformation of the target region and deliver a 
better, more focused treatment. 

In accordance with the invention, it may also be desirable to acquire several 
preoperative three-dimensional (tomographic) images of the target region and obtain 
preoperative information about the deformation or squeezing of the target region during 

15 the respiratory cycle. In particular, each such squeezing or deformation state of the 

tumor is characterized by a specific relative placements of the fiducial markers. During 
the operation, this relative placement of the fiducials may be used to compute the current 
deformation state of the target region at individual time points so that the target region 
may be compensated for in accordance with the invention since this deformation 

20 information may be readily incorporated into the motion pattern correlating the external 
to internal motion as described above. 
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la some cases, it is desirable to use a simplified model for the correlation between 
the internal and the external motion, i.e. motion of the target and motion of the skin 
surface, since that motion is fairly easily modeled. Typically, such a model for motion 
correlation in accordance with the invention is computed from several image pairs 
5 showing the current placement of the target at the corresponding time points and a series 
of matching sensor readings for the placement of the patient's skin surface. However, in 
accordance with the invention, the model for the motion correlation can also consist of a 
single time point where the patient is instructed to return to the position corresponding to 
this time point and where irradiation only takes place while the patient's respiratory state 
10 corresponds to this position as controlled by the computer system. 

To address patient shift or a systematic drift of the patient during irradiation in 
accordance with the invention, the deformation model representing correlation between 
the internal and the external motion must be recomputed during treatment. In some 
cases, it is undesirable to interrupt the treatment for such recomputation. To avoid such 
15 interruption, the system may generate a very small number of snapshots (showing both 
external and internal sensor readings). During treatment, new snapshots (showing both 
sensor readings) may be generated. The system may then perform a continuous real-time 
update of the deformation model. Thus, for each new snapshot, the entire model for the 
correlation is recomputed based on a small set of the most recent snapshots. 

20 In accordance with another embodiment of the invention, a general model for the 

correlation between the internal and external markers may be generated. As described 
above, a specific model for the correlation between internal and external motion for each 
individual patient is generated. In some cases, the necessity of using internal fiducial 
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markers can be obviated by determining a general model for the correlation between the 
internal motion and the external motion from several patients and earlier treatment data. 
Using this past data, the motion of certain anatomic regions under respiration is 
represented by a general model since the movement of those anatomic regions does not 
5 change between patients. Then, since the general model is being used, the system only 
observes the external motion of the target region at periodic time intervals during 
treament since the internal motion may be inferred from the general model and the 
occasional updates. 

In accordance with another embodiment of the invention, the system may 
10 compensate for system time delays. In particular, in some cases, the therapeutic beam is 
too slow to follow the motion of the target region since the speed of the therapeutic beam 
is limited by the mechanical properties of that therapeutic beam. In such cases, the data 
transfer between the device moving the treatment beam is not sufficiently fast to place 
the beam at the desired position with the desired speed. In this case, the inferred 
15 placement of the target is correct, but a systematic lag between actual and commanded 
placement of the therapeutic beam will occur which decreases the overall accuracy of the 
treatment. This problem is overcome by the system in accordance with the invention. In 
particular, based on our known model for the correlation between the internal motion and 
the external motion, the system can predict the proper placement of the internal target 
20 region. Thus, the target region location at various times during the respiratory cycle or 
other motion may be determined ahead of time. In addition, the system may determine 
the system time lag. Thus, the system may use the anticipated movement of the target 
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region and the known time lag to compensate for the time lag by anticipating that time 
lag. 

While the foregoing has been with reference to a particular embodiment of the 
invention, it will be appreciated by those skilled in the art that changes in this 
5 embodiment may be made without departing from the principles and spirit of the 
invention, the scope of which is defined by the appended claims. 
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Claims : 

1 . A method for compensating for breathing and other motions of the patient 
during treatment, the method comprising: 

generating images of the target region prior to the treatment; 
5 periodically generating positional data about the internal target region; 

continuously generating positional data about external motion of the patient's 
body using one or more external sensors; and 

generating a correspondence between the position of the internal target region 
and the external sensors so that the treatment is directed towards the position of the target 
10 region of the patient based on the positional data of the external sensors and the target 
region's position is subsequently matched to the position of the target region in the 
preoperative images. 

2. The method of Claim 1, wherein periodically generating positional data 
comprises generating positional data about one or more fiducial markers located near the 

15 target region and wherein generating a correspondance further comprises computing the 
deformation state of the target region based on the relative position of the fiducial 
markers. 

3 . The method of Claim 2 further comprising preoperative planning the 
treatment of the patient, the preoperative planning further comprising adding the 

20 computed deformation of the target region into preoperative planning based on the 
acquisition of more than one tomographic data set. 



4 
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4. The method of Claim 1, wherein generating a correspondance further 
comprises generating a deformation model for representing the correlation between the 
external and the internal patient motion further comprising generating a single snapshot 
so that the patient is instructed to return to the position within the respiration cycle 

5 corresponding to this single point. 

5. The method of Claim 1, wherein generating a correspondance further 
comprises generating a deformation model for representing the correlation between the 
external and the internal patient motion further comprising generating a series of 
snapshots to obtain a model for the correlation between the internal and the external 

10 motion, generating a new intraoperative snapshot, continuously updating the model 
during treatment by recomputing the entire model when the intraoperative snapshot is 
generated. 

6. The method of Claim 1, wherein generating images prior to treatment 
further comprising gathering a series of treatment data from more than one patient, 

15 generating a general model from the series of treatment data for the motion correlation 
between the external and the internal motion wherein the general model is used to one or 
more of infer the internal position data at fewer time points and without using internal 
fiducial markers. 

i 

7. The method of Claim 1 further comprising moving a treatment beam in 
20 response to the correspondance of the internal and external motion. 
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8. The method of Claim 1 further comprising moving the individual leaves 
of a multileaf collimator dynamically in response to the correspondance of the external 

* 

and internal motions. 



9. The method of Claim 1 further comprising gating the treatment in 
5 response to the correspondance between the internal and external motions so that the 
treatment is switched on and off periodically. 



1 0. The method of Claim 7, wherein the treatment beam moving further 
comprises compensating for the time lag between a command to move the treatment 
beam and the movement of the treatment beam by determining the placement of the 
10 target region at a future time from the deformation model. 



1 1 . The method of Claim 10, wherein the compensation further comprises 
determining the periodicity of the pulsation motion and the respiration motion. 



12. The method of Claim 1 further comprising distinguishing between 
voluntary patient motion and respiratory motion by comparing an actual internal 
15 placement of the target region to a placement of the target region inferred from a 

♦ 

deformation model and recomputing the deformation model whenr the deviation between 
actual internal placement of the target region and the inferred placement of the target 
region exceeds a threshold value. 



13. The method of Claim 1 further comprising delineating the target region 
20 shown in an image generated prior to the treatment and matching the delineation to the 
three dimensional shape of the target region shown in the images generated prior to the 
treatment in order to determine the position of the target region. 
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14. The method of Claim 13, wherein the delineating further comprises 
projecting the three dimensional shape of the target region generated by the images 
generated prior to treatment onto the periodically generated positional data. 

15. An apparatus for compensating for breathing and other motions of the 
patient during treatment, the apparatus comprising: 

a first detection device for periodically generating positional data about the 
internal target region; 

a second detection device for continuously generating positional data about 
pulsation motions within the patient; and 

a processor that receives the positional data about the internal target region and 
the pulsation in order to generate a correspondence between the position of the internal 
target region and the pulsation so that the treatment is directed towards the position of 
the target region of the patient based on the positional data obtained from the pulsation 
sensors. 

16. The apparatus of Claim 15 further comprising a third sensor for 
continuously generating positional data about respiratory motion and wherein the 
processor receives the positional data for the internal target region, the respiratory sensor 
and the pulsation sensor to compensate for the respiration and pulsations motions of the 
patient. 

17. The apparatus of Claim 1 5 further comprising a treament beam having a 
multiieaf collimator wherein the individual leaves of the multileaf collimator are moved 
dynamically in response to the correspondance of the external and internal motions. 
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1 8. The apparatus of Claim 1 5 further comprising a treatment beam having a 
gating element so that, in response to the correspondance between the internal and 
external motions, the treatment beam is switched on and off periodically. 

19. A method for compensating for breathing and other motions of the patient 
5 during treatment, the method comprising: 

generating images of the target region prior to the treatment; 
periodically generating ultrasound images indicating positional data about the 
internal target region; 

determining the current position of the internal target region based on the 
10 ultrasound images so that the treatment is directed towards the position of the target 
region of the patient based on the ultrasound image. 

20. The method of Claim 19 further comprising repositioning an ultrasound 
camera in order to make the target region visible in the ultrasound images. 

21 The method of Claim 19, wherein the repositioning further comprises 
15 remotely operating a robotic arm connected to the ultrasound camera in order to 
reposition the camera. 

22. The method of Claim 21, wherein generating ultrasound images further 
comprises preventing the ultrasound device or robotic arm from applying undesirable 
force to the patient using a force sensor. 
20 23 . The method of Claim 2 1 , wherein generating the ultrasound images 

further comprises positioning a liquid filled cushion between the ultrasound device and 
the patient to improve the ultrasound pictures. 
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24. The method of Claim 21, wherein generating the ultrasound images 
further comprises positioning a material filled cushion between the ultrasound device and 
the patient to improve the ultrasound pictures. 
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